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ABSTRACT 



As a component of a consulting engagement with a Midwestern 
comprehensive mental health center, an Outpatient Assessment Questionnaire 
(OAQ) was administered pre-post to 124 outpatient clients in a three month 
time series design. These comparisons yielded significant changes on thirteen 
of the twenty dimensions, including general affect, positive affect, negative 
affect, anger, depression, coping via escape methods, level of functioning, 
mental coping methods, mental health, psychosis, seek professional help, 
spouse perception, and stress perception. Results show the greatest amount of 
improvement on the scales that measure emotional dimensions (affect, negative 
and positive affect, anger, and depression) . Since most of the clients had 
presented complaints of depression and stress-related emotional problems, it 
appears that therapy over the course of six weeks had its greatest effect on 
the kinds of problems these clients were experiencing. In addition, clients 
showed improvement in the use of mental methods of coping with stress and 
tended not to escape as much as a means of coping. Their level of daily 
functioning improved slightly and they reported fewer symptoms of mental 
health distress, personal stress, and psychotic symptoms. There was a slight 
improvement in perception of their spouses. At the end of therapy, they were 
more likely to seek professional help. No changes were observed on the seven 
psycho-social dimensions. (MKA) 
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ABSTRACT 



As a component of a consulting engagement with a Midwestern 
comprehensive mental health center, an Outpatient Assessment 
Questionnaire (OAQ) was administered pre-post to 124 outpatient clients 
in a three month time series design. The OAQ measured changes in 
responses on twenty psychosocial dimensions. These comparisons yielded 
signiHcant changes on thirteen of the twenty dimensions including 
general affect, positive affect, negative affect, anger, depression, 
coping via escape methods, level of functioning, mental coping methods, 
mental health, psychosis, seek professional help, spouse perception and 
stress perception. 
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INTRODUCTION 



This consultation was a program evaluation of a regional Midwest community 
mental health center’s services to their clients. The evaluation was a private consultation 
provided by Lloyd & Associates, a psychological consulting organization. This 
evaluation was the second part of a three phase consultation project that included the 
development of a mental health evaluation questionnaire, the evaluation of therapy 
effectiveness in producing positive changes in clients and the training of mental health 
staff in conducting their own future program evaluations using the instrument. The 
purpose of the first phase was to develop a mental health questionnaire that would 
reliably differentiate between normal non patients, chronic patients and outpatients. The 
mental health center primarily wanted an instrument they could use in evaluating the 
effectiveness of their outpatient services. The first phase resulted in the development of a 
mental health questionnaire containing 13 mental health scales and 7 psycho-social 
coping scales that reliably differentiated between the three comparison groups. This 
questionnaire is titled The Outpatient Assessment Questionnaire (OAQ) and a copyright 
is currently pending. The second phase of this consultation involved using the OAQ to 
evaluate the effectiveness of the outpatient mental heath services provided by the 
Community Counseling Center. It was expected that clients seeking outpatient therapy 
would show positive change on scales of the OAQ over the courses of six weeks of 
outpatient therapy. 
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METHOD 



Subjects: 

Subjects in this evaluation were 124 adult outpatient clients from a large Midwest 
community mental health center that provides comprehensive mental health services. 
These clients were evaluated at intake not to be Chronic and therefore were treated on an 
outpatient basis. 

Instrument: 

The evaluation instrument was the Outpatient Assessment Questionnaire (OAQ) 
which was developed and validated in house for the purpose of evaluating the outpatient 
treatment services. The OAQ is a 75 item questionnaire containing Likert type items that 
provide measures on 13 clinical scales and 7 psycho-social coping scales. The instrument 
has demonstrated adequate validity and reliability in a previous pilot study. The 13 
clinical scales include: 

1 General Affect Scale measures degree of negative and positive affect. 

a. Negative Affect Subscale measures negative feelings. 

b. Positive Affect Subscale measures positive feelings. 

2. Anger Scale measures degree of anger control of depression and life 
satisfaction. 

3. Depression Scale measures degree and frequency depressed feelings and 
behavior. 

4. Interpersonal Skill Scale measures ease and comfort of interpersonal 
interaction. 

5. Level Of Functioning Scale measures degree of independent daily 
functioning. 

6. Mental Health Scale measures frequency of negative mental health 
symptoms. 

7. Psychosis Scale measures frequency of psychotic symptoms. 

8. Social Support Scale measures frequency and comfort of talking about 
personal problems with other people. 

9. Spouse General Perception Scale measures degree of satisfaction with 
spouse and family. 

10. Spouse Relationship Scale measures frequency and degree of positive 
interactions with spouse. 

11. Life-Stress Perception Scale measures degree of satisfaction with 
personal affairs and health. 

The seven psycho-social coping scales include: 

1. Behavioral Coping measures use of behavior methods of coping. 

2. Escape measures use of escape as a means of coping. 
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3. Injure Self measures suicide attempts. 

4. Injure Others measures attempts to injure other people. 

5. Mental Coping measures use of cognitive coping and relaxation methods. 

6. Seeking Professional Help measures comfort in seeking professional 
help. 

7. Substance Abuse measures frequency and degree of substance use. 

Design & Procedure: 

This was a pre - post repeated measures design in which clients were given the 
OAQ at intake and again at the end of six weeks of therapy. When new clients arrived 
they were given a folder containing intake forms and the OAQ and an informed consent 
form. The receptionist instructed each new client to “Please complete the following 
paper work as part of our routine admission procedures” . Clients completed the 
material at a table reserved for filling out confidential materials and returned the materials 
to the receptionist when finished. The receptionist assured the clients that the 
information was confidential and asked if the client had any questions about the 
procedure. The receptionist printed the client’s case number on both the face sheet and 
the OAQ and then attached the face sheet to the OAQ. Questionnaires were stored in the 
intake clerk’s filing cabinet until the Center’s computer data entry clerk entered the data 
into a spread sheet program, giving each client a new ID number. At the end of six weeks 
clients were either given, or mailed a letter that read: 

“Dear ( Client Name), 

We are following up with you to see how you are doing. Please put your name on 
the form we have enclosed and answer some questions for us about how you have been 
doing. Find a quiet place to complete the questionnaire and the informed consent form 
by yourself. Read the instructions carefully and fill out all the information. The 
information you give us will be kept confidential and will be used to help us improve our 
services. Please put the questionnaire and form into the envelope we have provided and 
return it to us through the mail. There is no need for you to add postage. We are happy 
to have been able to provide a service to you. ” 

The letter contained the OAQ with the client’s case number printed on the of the 
form, a brief client satisfaction survey, informed consent form and a self addressed return 
envelope. The instructions on the OAQ read as follows: “Please read the questions and 
the corresponding response scales. Please answer all questions to the best of your 
recollection. Your answers provide important information to your therapist for treatment 
and assessment purposes. Your answers on this questionnaire are confidential to the 
professional staff. Thank you. ” The evaluation period ran for three months during which 
time 124 outpatient clients completed both pre and post questionnaires. 
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RESULTS 



Table 1 shows the pre and post test means, standard deviations and t- test values 
for pre-post differences on each of OAQ dimensions. Significant changes were noted in 
13 of the 20 dimensions. The 13 dimensions that were observed to significantly change 
for the better include the General Affect, Negative Affect, Positive Affect, Anger, 
Depression, Escape, Level of Functioning, Mental Coping, Mental Health, Psychosis, 
Seeking Help, Spouse Perception and Stress Perception Scales. Those Scales that did 
not manifest significant changes were Behavior Coping, Injure Others, Injure Self, 
Interpersonal Skills, Social Support, Spouse relations and Substance Abuse. 

The first dimension, the General Affect Scale is comprised of two sub scales. 
Negative and Positive Affect. The total Affect Scale is a five point Likert type scale in 
which clients rate their negative and positive feelings. After reversing the positive affect 
sub scale to be congruent with desirable change in the negative affect sub scale, where 
higher values correspond with improvement, a significant positive change from pre test to 
post test means (2.8 vs. 3.0) in total affect was noted at the end of six weeks of therapy. 
Most of this change was due changes in negative affect in which clients reported fewer 
negative emotions. The Positive Affect Scale, (unreversed) in which lower values 
correspond to desirable change, also significantly changed with an increase in slightly 
more positive emotions following therapy. Another emotional dimension. Anger, also 
changed for the better. The Anger Scale is a four point Likert rating scale in which higher 
values indicate less anger. The change from pre test mean of 2.43 to post test mean of 2.6 
was also significant. The emotional dimension of Depression was measured on a five 
point Likert type rating scale. On this scale higher values correspond with more 
depression. The change from a pre test mean of 3.2 to a post test mean of 2.8 was 
significant indicating clients were somewhat less depressed after therapy. Since the most 
common presenting complaints were depression and stress related emotions it appears 
that therapy is associated with desirable changes in these emotional dimensions. 

The Escape dimension as measured on a four point Likert Scale, significantly 
decreased, indicating clients appeared to use escape less as a means of coping with stress 
following treatment. Level of functioning also demonstrated slight improvement. This 
scale was a four point Likert Scale where lower values are associated with improved 
daily functioning. The change from pre to post test means (2.11 vs. 1.96) was slight but 
significant. The Mental Coping and Mental Health Scales also showed significant 
improvement from pre to post test. Both of these scales are four point Likert scales where 
higher values correspond to desirable change. Following therapy clients were using more 
active mental coping methods and reporting fewer negative mental health symptoms than 
before therapy. Clients also reported fewer occasions of experiencing psychotic 
symptoms following therapy. The Psychotic Scale was also a four point Likert scale 
where higher values indicate fewer psychotic symptoms. The change from pre test mean 
of 3.28 to post test mean of 3.51 was significant. In addition, clients also appeared more 
likely to seek professional help after therapy than before. This four point Likert scale 
showed a slight but significant increase ( 2.26 vs. 2.54) in willingness to seek 
professional help following therapy. Clients also reported more positive perceptions of 
their spouse, if married, following therapy. The Spouse Perception Scale is a four point 
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Likert scale where lower values represent more positive perceptions of spouse. Means on 
this scale significantly changed from pre mean of 2.18 to post mean of 2.01. The last 
dimension to demonstrate improvement was the stress perception scale. On this four 
point Likert scale lower values are associated with less perceived stress. The change 
from pre to post test (2.47 vs. 2.22) was significant. 

Seven dimensions did not manifest significant change from pre to post test. The 
behavior coping scale was a four point Likert scale in which higher values indicated 
greater use of behavioral methods of coping. The pre test mean of 2.79 was above the 
midpoint for this scale. Although the post test mean of 2.8 was a slight increase in the 
positive direction, it was not enough of a change to be considered statistically significant. 
The Injure Self and Injure Others scale was a simple 1. yes or 2. no response category. 
The pre test and post test means on both scales were very near 2 indicating nearly all 
clients who answered these questions had not tried to injure other people nor had suicide 
attempts in the last three months. The Interpersonal Skills scale was a three point Likert 
scale with higher values indicative of greater ease in interpersonal interactions. Both pre 
and post test means are relatively high (1.94 vs. 1.92) and nearly identical. The Social 
support scale was a four point Likert scale where lower values indicate fewer sources of 
social support. Again, the pre and post test means (2.48 vs 2.41) on this scale are nearly 
identical and are in the midpoint area on the scale. There was also no significant change 
on the spouse relations scale. On this four point Likert scale lower values indicate better 
relations. The pretest mean of 1.74 was already relatively low and with little room for 
improvement the post test mean of 1.73 was almost identical. The last scale that showed 
no significant changes was the Substance Abuse Scale. This was comprised of a series of 
items asking clients to indicate frequency and degree of using legal (cigarettes/alcohol) 
and illegal substances (marijuana/cocaine). Response categories ranged from one to five, 
with lower values indicating little or no substance abuse. Again the pre and post test 
means are nearly identical (2.37 vs. 2.39) indicating no change in substance use over the 
course of therapy. 
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DISCUSSION 



The results indicate that clients showed improvement on 13 of the 20 psycho- 
social dimensions as measured on the OAQ. The greatest amount of improvement 
occurred on the scales that measured the emotional dimensions (Affect, Negative and 
Positive Affect, Anger, Depression). Since most of the outpatient clients seeking therapy 
at this regional mental health facility had presenting complaints of depression and stress 
related emotional problems it appears that therapy over the course of six weeks had it’s 
greatest effect on the kinds of problems these clients were experiencing. In addition, 
clients showed improvement in the use of mental methods of coping with stress and 
tended not to use escape as much a means of coping. Their level of daily functioning 
improved slightly and they reported significantly fewer symptoms of mental health 
distress, personal stress and psychotic symptoms. There was also a slight improvement in 
their perception of their spouse. At the end of therapy they were more likely to seek 
professional help for their personal problems than before. Perhaps this is because of the 
relief they were experiencing from therapy and the trust that developed over time with 
their therapists. 

On the seven psycho-social dimensions in which no change was observed to 
occur, this is probably due in part that the pre test means on many of these scales were 
initially in the positive area. For instance, on the Injure Self or Injure Others dimensions, 
the pre test means of 1.9 and 1.94 on a two point scale leaves little room for 
improvement. Similarly, the initial means on the dimensions of behavior coping, 
interpersonal skills, social support, spouse relations and substance abuse were initially in 
the positive area on their respective scales. 

In conclusion there is good evidence that the outpatient services provided to these 
clients had a positive effect on their mental health as measured by the OAQ. There is a 
question of degree or magnitude of effect. On some scales (Escape) the significant effect 
was meager while on other scales (Negative Affect) the effect was more impressive. It is 
possible that the effect of therapy was masked to some degree by the fact that this was an 
overall evaluation of client improvement without regard to the particular complaint or 
diagnostic category at the time of intake. It would be expected that clients seeking 
outpatient therapy would show greater positive changes on scales relating to their 
particular presenting problems than on scales that do not. It stands to reason that a client 
coming in for a stressful crisis would not show as much change on the depression 
dimension as would a client coming in with signs of clinical depression. Similarly a 
client with a good social support network would not be expected to change as much on 
the social support and interpersonal skills dimension as a client who comes in because he 
is lonely. By lumping all clients together to evaluate therapy effects without knowledge 
of their initial problem it could have masked the magnitude of therapy effects on the 
dimensions relevant to the particular problem. It is therefore recommended that in future 
evaluations, a new variable be added that codes the client’s initial complaint or diagnosis. 
Then by grouping clients together with the same complaints (e.g. Depression) we can 
more accurately assess the degree of impact on OAQ dimensions that are relevant to the 
particular complaint. 
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Perhaps the most significant threats to internal validity in this evaluation are 
history, statistical regression, testing and placebo effects. Since this was a repeated 
measures design over six weeks with people in need, these threats can not be completely 
ruled out. Perhaps in the future using a pre-post cyclical cohort design which allows both 
within and between groups comparisons with clients who arrive at different cycles, these 
threats may be minimized. 
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TABLE 1 



PRE - POST COMPARISON OF 20 DIMENSIONS OF OAQ 



paSMiiHi 
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1 1 


Affect 


122 


2.8 (.385) 


3.0 (.426) 


-5.09** 


Negative Affect 


122 


2.5 (.837) 


3.14 (.862) 


-7.61** 


Positive Affect 


122 


3.13 (.700) 


2.8 (.703) 


4.50** 


Anger 


122 


2.43 (.459) 


2.6 (.486) 


-4.03** 


Behavioral Coping 


120 


2.72 (.697) 


2.8 (.754) 


-1.92 


Depression 


122 


3.2 (.871) 


2.8 (.837) 


5.16** 


Escape 


121 


2.7 (.981) 


2.5 (.913) 


2.05* 


Injure Others 


122 


1.98 (.089) 


1.98 (.240) 


0.00 


Injure Self 


120 


1.90 (.301) 


1.94 (.235) 


-1.22 


Interpersonal Skills 


124 


1.94 (.603) 


1.92 (.610) 


0.28 


Level of Functioning 


124 


2.11 (.599) 


1.96 (.708) 


2.59* 


Mental Coping 


120 


2.34 (.742) 


2.65 (.637) 


-4.30** 


Mental Health 


122 


2.81 (.857) 


3.14 (.821) 


-4.04** 


Psychosis 


120 


3.28 (.634) 


3.51 (.512) 


-4.11** 


Seek Professional Help 


119 


2.26 (1.16) 


2.54 (1.15) 


-2.63** 


Social Support 


124 


2.48 (.679) 


2.41 (.743) 


1.30 


Spouse Perception 


121 


2.18 (1.03) 


2.01 (.969) 


2.28* 


Spouse Relations 


121 


1.74 (.637) 


1.73 (.665) 


0.22 


Stress Perception 


123 


2.47 (.710) 


2.22 (.657) 


4.25** 


Substance Use 


76 


2.37 (.514) 


2.39 (.447) 


-0.42 



* P < .05 
** P < .01 
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